Transformer Circuits

Researchers challenge the established belief that induction heads require two layers in transformer architectures, arguing that the phenomenon may be attributable to two attention heads rather than tw…